Motion sickness estimation device, motion sickness prevention device, and motion sickness estimation method

ABSTRACT

A motion sickness estimation device includes: a line-of-sight oscillation detector to detect oscillation of a line of sight of an occupant by using an imaging device; an object oscillation detector to detect oscillation of an object located in the line of sight; an oscillation comparator to calculate an oscillation difference from the oscillation of the line of sight and the oscillation of the object; and a motion sickness determiner to determine, on a basis of the oscillation difference, whether the occupant is in a motion sickness state.

TECHNICAL FIELD

The present invention relates to a device and method for estimatingmotion sickness of an occupant in a conveyance.

BACKGROUND ART

From old times, occupants who read documents, such as newspapers orbooks, or watch images through displays in moving vehicles have beentroubled by motion sickness resulting from being subjected to vibrationof the vehicles due to the road surface conditions, engines, railjoints, or the like. Motion sickness is also called “kinetosis” or“acceleration sickness”, and is an autonomic response caused byvibration, in particular irregular repetition of acceleration anddeceleration, stimulating the semicircular canals and vestibule of eachinner ear. To prevent motion sickness, occupants have taken measures,such as looking in the distance through a window or taking anti-motionsickness drugs.

It is thought that motion sickness also occurs when a conflict occursbetween body balance sense information obtained from the humansemicircular canals and visual information obtained by the human eyes.Based on this, studies have been conducted that aim to resolve symptomsof motion sickness by showing an occupant an image consistent with thebalance sense information (see Non Patent Literature 1).

As a method for preventing motion sickness, there has been proposed amethod that makes it possible to prevent motion sickness by estimatingmotion sickness of an occupant on the basis of an integral value of theacceleration of the vehicle or the acceleration of the head of theoccupant and controlling the vehicle to reduce the acceleration of thehead of the occupant. Since it is difficult to attach a sensor to thehead, the acceleration of the head has been estimated from posture data,a pressure distribution, and the like (see Patent Literature 1).

CITATION LIST Patent Literature

Patent Literature 1: Japanese Patent Application Publication No.2007-236644 (pages 4-6 and FIG. 2)

Non Patent Literature 1: Etsuji Kitagawa, “Research and Development onSystem for Reducing Kinetosis in Using Media in Conveyance,” The Journalof The Institute of Image Information and Television Engineers, Vol. 67,No. 11, pp. J388-J399, 2013

SUMMARY OF INVENTION Technical Problem

However, the technique against motion sickness as in Patent Literature 1only detects the acceleration of the head. This technique cannotcompensate a difference, which is one of the causes of motion sickness,between oscillation of the line of sight due to vestibulo-ocular reflexor optokinetic nystagmus and oscillation of an object located in theline of sight, and cannot detect vestibulo-ocular reflex. Also, in themethod of Non Patent Literature 1, it is necessary to force specialactions on the occupant, such as force the occupant to watch a terminalwhile limiting the viewing conditions, such as the distance to theterminal.

The “vestibulo-ocular reflex” is a reflex function in which the line ofsight moves in a direction opposite to a direction in which the head isaccelerated. For example, a function in which when a head is tilted upduring looking ahead, the line of sight involuntarily moves downward isvestibulo-ocular reflex. When the view remains unchanged while thesemicircular canals sense acceleration on a watercraft or in a train, aconflict occurs between balance sense information and visualinformation, causing motion sickness.

The “optokinetic nystagmus” refers to an oscillation state of theeyeballs caused by repetition of slow eyeball movement and fast eyeballmovement. It occurs, for example, in looking out of a window of a trainor in similar situations, due to alternation of a slow eye movement thatfollows an object sequentially passing before the eyes and a fastmovement in the opposite direction that tries to catch the next object.Optokinetic nystagmus causes abnormal actions of the extraocularmuscles. It is thought that nerve signals caused by the actionsstimulate the medulla oblongata, which controls the autonomic nervoussystem, through the vestibular nerves and disrupt its function, causingsymptoms such as nausea.

The present invention has been made to solve the problems as describedabove, and is intended to provide a motion sickness estimation devicecapable of accurately estimating motion sickness by detectingoscillation of a line of sight without having to force a special action,such as watching a terminal, on an occupant.

Solution to Problem

To achieve the above object, according to the present invention, thereare provided a line-of-sight oscillation detector to detect oscillationof a line of sight of an occupant, an object oscillation detector todetect oscillation of an object located in the line of sight, anoscillation comparator to calculate an oscillation difference from theoscillation of the line of sight and the oscillation of the object, anda motion sickness determiner to determine, on a basis of the oscillationdifference, whether the occupant is in a motion sickness state.

Advantageous Effects of Invention

With this configuration, it is possible to provide a motion sicknessestimation device that has no need to force a special action, such aswatching a terminal, on an occupant, by the line-of-sight oscillationdetector detecting oscillation of a line of sight of the occupant in aconveyance, the object oscillation detector detecting oscillation of anobject located in the line of sight of the occupant, the oscillationcomparator calculating an oscillation difference from the oscillation ofthe line of sight and the oscillation of the object, and the motionsickness determiner determining, on a basis of the oscillationdifference, whether the occupant is in a motion sickness state.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a motionsickness prevention device including a motion sickness estimation deviceaccording to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram illustrating an arrangement of elementsin a vehicle including the motion sickness prevention device accordingto the first embodiment of the present invention.

FIG. 3 is a block diagram illustrating another configuration of themotion sickness prevention device according to the first embodiment ofthe present invention.

FIG. 4 is a flowchart illustrating a motion sickness estimation methodaccording to the first embodiment of the present invention.

FIG. 5 is a block diagram illustrating a configuration of a motionsickness prevention device including a motion sickness estimation deviceaccording to a second embodiment of the present invention.

FIG. 6 is an explanatory diagram illustrating an arrangement of elementsin a vehicle including the motion sickness prevention device accordingto the second embodiment of the present invention.

FIG. 7 is a flowchart illustrating a motion sickness estimation methodaccording to the second embodiment of the present invention.

FIG. 8 is a block diagram illustrating a configuration of a motionsickness prevention device including a motion sickness estimation deviceaccording to a Gccond third embodiment of the present invention.

FIG. 9 is an explanatory diagram illustrating an arrangement of elementsin a vehicle including the motion sickness prevention device accordingto the third embodiment of the present invention.

FIG. 10 is a flowchart illustrating a motion sickness estimation methodaccording to the third embodiment of the present invention.

DESCRIPTION OF EMBODIMENTS First Embodiment

FIG. 1 is a block diagram illustrating a configuration of a motionsickness prevention device including a motion sickness estimation deviceaccording to a first embodiment. A motion sickness prevention device 10includes a motion sickness estimation device 100 and a vehiclecontroller 160. The motion sickness estimation device 100 includes aline-of-sight oscillation detector 110, an object oscillation detector120, an oscillation comparator 130, a determination information storage140, a motion sickness determiner 150, and a warning presenter 170. Theline-of-sight oscillation detector 110 detects oscillation of a line ofsight of an occupant. The object oscillation detector 120 detectsoscillation of an object at which the occupant is looking. Theoscillation comparator 130 compares the amount of oscillation of theline of sight that is an output of the line-of-sight oscillationdetector 110 and the amount of oscillation of the object located in theline of sight that is an output of the object oscillation detector 120.The determination information storage 140 stores a threshold value of anoscillation difference that can lead to motion sickness. Here, theoscillation difference that can lead to motion sickness is a differencebetween the amount of oscillation of the line of sight that is an outputof the line-of-sight oscillation detector 110 and the amount ofoscillation of the object located in the line of sight that is an outputof the object oscillation detector 120. Here, the threshold value is thevalue of the oscillation difference above which motion sickness can becaused. In other words, the threshold value is the value of theoscillation difference at or below which motion sickness will not becaused. The motion sickness determiner 150 compares an output of theoscillation comparator 130 with the threshold value of the determinationinformation storage 140 and determines a motion sickness state. Thevehicle controller 160 controls the vehicle on the basis of adetermination result output by the motion sickness determiner 150. Thewarning presenter 170 issues a warning to the occupant and prompts theoccupant to take a rest, on the basis of the determination result outputby the motion sickness determiner 150. The warning may be a warningusing a sound through a speaker, or may be a warning using an imagedisplayed on an in-vehicle display, a head-up display (HUD), aninstrument panel, or the like. The “head-up display (HUD)” is a displaythat directly displays information in the field of view of a person byprojecting an image onto a transparent optical glass element or othermethods. The “instrument panel” is a control panel located in front of adriver's seat of an automobile.

In the line-of-sight oscillation detector 110, to detect oscillation ofthe line of sight, movement of the position of the iris in an imaged eyeis detected as the oscillation. The “oscillation of the line of sight”is oscillation of a visual axis that is a line connecting a center of aneye and a target being viewed, and is a phenomenon caused by oscillationof the target being viewed. Methods for detecting oscillation of theline of sight include electrooculography, the optical lever method, thesearch coil method, the limbus tracking method, the corneal reflectionmethod, and the like. The “electrooculography” focuses on the fact thatchange in voltage of an eyeball is substantially proportional to therotation angle of the eyeball, and places skin electrodes around the eyeand measures movement of the eyeball from change in voltage of theeyeball. The “optical lever method” is a method that places a contactlens with a small mirror attached to its edge on the cornea and extractsreflected light resulting from reflection of a light beam by the mirrorby using image analysis or photoelectric conversion. The “search coilmethod” is a method that attaches coils to the edge of a contact lens,places a person wearing the lens in a uniform alternating magneticfield, and extracts an induced current that is proportional to rotationof the eyeball, thereby detecting the line-of-sight oscillation. The“limbus tracking method” is a method that measures eyeball movement byradiating weak infrared light to the boundary between the iris and thesclera and detecting the resulting reflected light with a sensor. The“corneal reflection method” is a method that measures eyeball movementon the basis of the position of a corneal reflection image that brightlyappears when the cornea is illuminated with illumination light from apoint light source. The present invention does not require attaching anobject for measurement to the eyeball as described above.

The present invention detects oscillation of the line of sight byimaging line-of-sight oscillation of the occupant. In detectingoscillation of the line of sight, the iris is detected in an imaged eye,for example. Then, it is detected whether the boundary between theposition of the iris and the position of a part, such as the sclera,outside the iris is moving over time. When it is moving, it isdetermined that there is line-of-sight oscillation. For example, whenchange in frequency of movement of a moving position of the boundary islarge, it can be determined that the oscillation amount is large. Indetecting the direction of the oscillation, it is possible to take, as areference, the mounting direction in which a camera for imagingline-of-sight oscillation of the occupant is mounted. It is alsopossible to attach an acceleration sensor to a camera for imagingline-of-sight oscillation of the occupant and detect a displacement ofthe camera from a vertical direction. As above, it is possible to matchthe reference of the direction of the detection of the line-of-sightoscillation by imaging with the camera and the reference of thedirection of the detection of oscillation of the object by anacceleration sensor.

FIG. 2 is an explanatory diagram illustrating an arrangement of elementsin the vehicle including the motion sickness prevention device accordingto the first embodiment of the present invention. In the vehicleillustrated in FIG. 2, an in-vehicle camera 210, an in-vehicle display220 with an acceleration sensor attached thereto, an oscillationcomparator 230, a storage device 240, a determiner 250, a conveyancecontroller 260, and a speaker 270 are arranged. The correspondencerelationship between the elements in FIG. 2 and the elements in FIG. 1is as follows: The in-vehicle camera 210 corresponds to theline-of-sight oscillation detector 110. The in-vehicle display 220 withthe acceleration sensor attached thereto corresponds to the objectoscillation detector 120. The oscillation comparator 130 corresponds tothe oscillation comparator 230. The storage device 240 corresponds tothe determination information storage 140. The determiner 250corresponds to the motion sickness determiner 150. The conveyancecontroller 260 corresponds to the vehicle controller 160. The speaker270 corresponds to the warning presenter 170.

The in-vehicle camera 210 is a device that obtains line-of-sightoscillation of an occupant, which is a detection target, by imaging. Thein-vehicle camera 210 is connected to the oscillation comparator 230.The camera illustrated in FIG. 2 may be connected through wiring, suchas wire harness, or may be connected wirelessly. The camera may be acommon RGB camera, or may be an IR camera. The “RGB camera” is a camerathat communicates signals of three colors of red, green, and bluethrough three different cables or the like, and typically uses threeindependent CCD sensors. The “IR camera” is an infrared camera, and acamera sensitive to wavelengths in the infrared region.

In FIG. 2, the acceleration sensor is provided to the in-vehicle display220, which is assumed to be viewed by the occupant for a long time.Since the in-vehicle display 220 is located near the occupant, theapparent amount of movement thereof due to vibration is large comparedto a distant object, and thus the above-described difference from theoscillation of the line of sight is large. Thus, watching the in-vehicledisplay 220 is likely to cause motion sickness. Also, it is easy todirectly attach the acceleration sensor to the in-vehicle display 220.From the above, by attaching the acceleration sensor to the in-vehicledisplay 220, it is possible to accurately measure the oscillation anduse it for motion sickness estimation. The position where theacceleration sensor is mounted is not limited to the in-vehicle display220.

The oscillation comparator 230 in FIG. 2 compares the line-of-sightoscillation detected by the in-vehicle camera 210 and the oscillationdetected by the acceleration sensor provided in the in-vehicle display220. By calculating a difference between the line-of-sight oscillationdetected by the in-vehicle camera 210 and the oscillation detected bythe acceleration sensor provided in the in-vehicle display 220, acomponent of vestibulo-ocular reflex, which causes motion sickness, isextracted. It is possible to time-integrate the difference, therebyproviding motion sickness estimation with reduced effect of noise.

The speaker 270 illustrated in FIG. 2 is provided to, when theoscillation difference obtained from the oscillation comparator 230exceeds the threshold value, determine that it is oscillation leading tomotion sickness, and issue a warning to the occupant. The motionsickness determination of the present invention is a determination thatthere is an indication of motion sickness, and the determination is madefrom the environment around the occupant, regardless of whether theoccupant is aware of being in a motion sickness state.

The conveyance controller 260 illustrated in FIG. 2 reduces theoscillation difference obtained from the oscillation comparator 230. Forexample, the conveyance controller 260 adjusts a suspension of a seat onwhich the occupant is sitting, to reduce sway of the person. The“suspension” of the seat of the vehicle has a function as a shockabsorber that prevents the unevenness of the road surface from beingtransmitted to the vehicle body, and is a mechanism that improves ridequality, handling stability, or the like. The conveyance controller 260also adjusts a suspension of a seat supporting the in-vehicle display.The conveyance controller 260 also performs deceleration to reduce sway.The conveyance controller 260 also presents a route with a good roadsurface condition by cooperating with a car navigation system. Theconveyance controller 260 also presents a point where it is possible totake a rest.

To enhance the effect of reducing motion sickness of the presentinvention, it is possible to change the environment in the vehicle,thereby reducing the occurrence of motion sickness of the occupant orreducing the degree of motion sickness. For example, changing theenvironment in the vehicle includes diffusing a fragrance substance intothe vehicle. Also, changing the environment in the vehicle includesplaying music for the purpose of relaxing the occupant or for otherpurposes. Also, changing the environment in the vehicle includes openingor closing a window for the purpose of exposing the occupant to theoutside air or for other purposes. Also, changing the environment in thevehicle includes adjusting an air conditioner for the purpose of makingthe occupant comfortable or for other purposes. Also, changing theenvironment in the vehicle includes changing the restraining state ofthe seat belt for the purpose of preventing the occupant from feelingpressure or for other purposes.

Although the above description has described the present invention asbeing implemented by hardware, part or all of the signal processingillustrated in FIG. 1 can be implemented by software or a programmedcomputer.

FIG. 3 is a block diagram illustrating another configuration of themotion sickness prevention device according to the first embodiment ofthe present invention. The illustrated motion sickness estimation deviceincludes a CPU 11, a program memory 12, a data memory 13, a bus 14 thatconnects these, an interface 15, and a warning presenter 16. The “CPU”,which is an abbreviation for Central Processing Unit, is a device thatperforms calculation or control.

The CPU 11 operates according to a program stored in the program memory12. It stores various data items into the data memory 13 during theprocess of the operation. An estimation result is supplied to thewarning presenter 16 through the interface 15. The estimation result isalso supplied to an operation section 17 of the vehicle through theinterface 15. The operation section 17 is, for example, a suspensionadjustment 17 a of a seat. The operation section 17 is a car navigationsystem 17 b. The operation section 17 is constituted by the suspensionadjustment or car navigation system, and may be constituted by one orboth of them. The operation section 17 corresponds to the vehiclecontroller 160 or conveyance controller 260 in the above description.

Hereinafter, the process performed by the CPU 11 will be described withreference to FIG. 4.

FIG. 4 is a flowchart illustrating a motion sickness estimation methodaccording to the first embodiment of the present invention. The motionsickness estimation method illustrated in FIG. 4 includes a travelingdetermination step S10, an estimation start signal generation step S20,an imaging step S30, a region extraction step S40, a line-of-sightoscillation detection step S50, an object oscillation detection stepS60, an oscillation comparison step S70, a motion sickness estimationstep S80, a vehicle control step S90, a motion sickness warning stepS100, and an engine stop determination step S110.

The traveling determination step S10 determines whether the vehicle isin a traveling state. The determination is made by acquiringdetermination as to whether the vehicle is traveling from CANinformation or the like. The “CAN”, which is an abbreviation forController Area Network, is a network standard for connecting anelectronic circuit and devices that has been developed as acommunication technique between in-vehicle devices.

When it is determined that the vehicle is in a traveling state, in theestimation start signal generation step S20, an estimation start signalis generated by a head unit and supplied to the in-vehicle camera 210and an illumination associated therewith. “Head unit” is a general termfor units that control audio devices, car navigation systems, or thelike. Next, in the imaging step S30, the in-vehicle camera 210, whichhas received the supplied signal, images an occupant.

In the region extraction step S40, a device associated with thein-vehicle camera 210 extracts a region of an eye of the occupant. Inthe line-of-sight oscillation detection step S50, the line of sight andits oscillation are detected from the image of the detected eye region.The result of the line-of-sight oscillation detection step S50 is sentto the oscillation comparison step S70. On the other hand, when it isdetermined in the traveling determination step S10 that the vehicle isnot in a traveling state, in the engine stop determination step S110, itis determined whether the engine is in a stopped state. When the vehicleis traveling, the motion sickness estimation is continued; when thevehicle is stopped, the estimation is not performed. When the vehicle isin a stopped state and the engine is also in a stopped state, theestimation ends.

After the estimation as to whether motion sickness is being experiencedis started at the estimation start signal generation step S20, in thein-vehicle monitor oscillation extraction step S60, oscillation of thein-vehicle monitor is extracted. In the oscillation comparison step S70,an oscillation difference is calculated from the oscillation of the lineof sight resulting from the line-of-sight oscillation detection step S50and the oscillation of the in-vehicle monitor resulting from thein-vehicle monitor oscillation extraction step S60. In the motionsickness estimation step S80, a motion sickness estimation is made fromthe oscillation difference on the basis of motion sickness estimationdata. In the vehicle control step S90, the operation section of thevehicle is controlled on the basis of the result of the motion sicknessestimation. Also, in the motion sickness warning step S100, a motionsickness warning is issued on the basis of the result of the motionsickness estimation. Then, the above flow is repeated by returning tothe traveling determination step S10.

The above has described the operations of the motion sickness estimationdevice and motion sickness estimation method. With the aboveconfiguration, by detecting the state of an occupant in a conveyance andoscillation of an object located in the line of sight of the occupantand comparing their oscillations by the estimation means, it is possibleto estimate the motion sickness state of the occupant. Further, by thevehicle controller controlling the vehicle on the basis of theestimation result, it is possible to prevent motion sickness of theoccupant.

Second Embodiment

FIG. 5 is a block diagram illustrating a configuration of a motionsickness prevention device including a motion sickness estimation deviceaccording to a second embodiment. A motion sickness prevention device 30includes a motion sickness estimation device 300 and a vehiclecontroller 360. The motion sickness estimation device 300 includes aline-of-sight oscillation detector 310, an object oscillation detector320, an oscillation comparator 330, a determination information storage340, a motion sickness determiner 350, and a warning presenter 370. Theline-of-sight oscillation detector 310 detects a line of sight of anoccupant and oscillation of the line of sight. The object oscillationdetector 320 images an object located in the line of sight detected bythe line-of-sight oscillation detector 310 and detects oscillation ofthe object. The oscillation comparator 330 compares the amount of theoscillation of the line of sight that is an output of the line-of-sightoscillation detector 310 and the amount of the oscillation of the objectlocated in the line of sight that is an output of the object oscillationdetector 320. An oscillation difference calculated by the oscillationcomparator 330 is input into the motion sickness determiner 350. Themotion sickness determiner 350 performs motion sickness determination onthe basis of a comparison of the oscillation difference with a thresholdvalue from the determination information storage 340. The thresholdvalue is the value of the oscillation difference above which motionsickness can be caused, as described in the first embodiment. Thevehicle controller 360 controls the vehicle on the basis of thedetermination result output by the motion sickness determiner 350. Thewarning presenter 370 issues a warning to the occupant and prompts theoccupant to take a rest or other actions, on the basis of thedetermination result output by the motion sickness determiner 350. Thewarning may be an audible warning or a visual warning, as in the firstembodiment.

FIG. 6 is an explanatory diagram illustrating an arrangement of elementsin the vehicle including the motion sickness prevention device accordingto the second embodiment of the present invention. In the vehicleillustrated in FIG. 6, an in-vehicle camera (for detecting a line ofsight) 410, an in-vehicle cameras (for imaging an object in the line ofsight) 420, an oscillation comparator 430, a storage device 440, adeterminer 450, a conveyance controller 460, and a speaker 470 arearranged. The correspondence relationship between the elements in FIG. 6and the elements in FIG. 5 is as follows: The in-vehicle camera 410corresponds to the line-of-sight oscillation detector 310. Thein-vehicle camera (for imaging an object in the line of sight) 420corresponds to the object oscillation detector 320. The oscillationcomparator 430 corresponds to the oscillation comparator 330. Thestorage device 440 corresponds to the determination information storage340. The determiner 450 corresponds to the motion sickness determiner350. The conveyance controller 460 corresponds to the vehicle controller360. The speaker 470 corresponds to the warning presenter 370.

The in-vehicle camera 410 is a device that obtains line-of-sightoscillation of an occupant, which is a detection target, by imaging theoccupant. The in-vehicle camera 410 is connected to the oscillationcomparator 430. The in-vehicle camera 420 is also connected to theoscillation comparator 430.

The in-vehicle camera 420 is one or more cameras located inside oroutside the vehicle. The in-vehicle camera 420 determines an objectlocated in the line of sight of the occupant obtained by the in-vehiclecamera 410, and obtains oscillation of the object from an imaged image.The determination of the object located in the line of sight obtained bythe in-vehicle camera 410 may be based on the position of a pupil.Alternatively, the determination of the object located in the line ofsight may be based on the orientation of the face. Alternatively, thedetermination of the object located in the line of sight may be made byreference to an image reflected on an eye. Since it is possible to makea comparison with oscillation of an arbitrary object, it is possible toestimate motion sickness of an occupant reading a book or an occupantoperating a mobile phone, for example. The in-vehicle camera 410 andin-vehicle camera 420 may be connected through wiring, such as wireharness, or may be connected wirelessly.

The oscillation comparator 430 compares the line-of-sight oscillationdetected by the in-vehicle camera 410 and the oscillation of the objectlocated in the line of sight obtained from the in-vehicle camera 420.The oscillation comparator 430 calculates a difference between theoscillations, thereby extracting oscillation due to vestibulo-ocularreflex.

The determination information storage 440 stores one or more thresholdvalues used in performing motion sickness determination from theoscillation difference output by the oscillation comparison device 430.

The motion sickness estimation device 450 can determine the degree ofthe motion sickness by comparing the oscillation difference output bythe oscillation comparison device 430 with the one or more thresholdvalues, corresponding to degrees of motion sickness, in thedetermination information storage device 440.

The conveyance controller 460 reduces the oscillation differenceobtained from the oscillation comparator 430. For example, theconveyance controller 460 adjusts a suspension of a seat on which theoccupant is sitting, to reduce sway of the person. The conveyancecontroller 460 also performs deceleration to reduce sway. The conveyancecontroller 460 also presents a route with a good road surface conditionby cooperating with a car navigation system. The conveyance controller460 also presents a point where it is possible to take a rest.

When the oscillation difference obtained from the oscillation comparator430 exceeds a threshold value, the speaker 470 determines that it is anoscillation leading to motion sickness, and issues a warning to theoccupant. It is also possible that a portion, such as the oscillationcomparator 430, connected to the speaker 470 determines, from acomparison of the oscillation difference with the threshold value, thatit is an oscillation leading to motion sickness, and the determinationresult is sent to the speaker 470.

Although the above has described implementation by hardware,implementation by software will be described.

As described for the motion sickness estimation method of the firstembodiment, a motion sickness estimation device for executing softwareof a motion sickness estimation method is illustrated by FIG. 3 asdescribed in the first embodiment; hereinafter, a process performed bythe CPU 11 will be described with reference to FIG. 7.

FIG. 7 is a flowchart illustrating the motion sickness estimation methodaccording to the second embodiment of the present invention. The motionsickness estimation method illustrated in FIG. 7 includes a travelingdetermination step S10, an estimation start signal generation step S20,an imaging step S30, a region extraction step S40, a line-of-sightoscillation detection step S350, an object imaging step S360, an objectoscillation detection step S365, an oscillation comparison step S70, amotion sickness estimation step S80, a vehicle control step S90, amotion sickness warning step S100, and an engine stop determination stepS110. For the reference characters of the steps of FIG. 7, the samereference characters are assigned to the steps that perform the sameoperations as those in the first embodiment.

The traveling determination step S10 determines whether the vehicle isin a traveling state. When it is determined that the vehicle is in atraveling state, in the estimation start signal generation step S20, anestimation as to whether motion sickness is being experienced isstarted. On the other hand, when it is determined in the travelingdetermination step S10 that the vehicle is not in a traveling state, inthe engine stop determination step S110, it is determined whether theengine is in a stopped state. When the vehicle is traveling, the motionsickness estimation is continued; when the vehicle is stopped, theestimation is not performed. When the vehicle is in a stopped state andthe engine is also in a stopped state, the estimation ends. When theestimation as to whether motion sickness is being experienced isstarted, an occupant is then imaged in the imaging step S30.

In the region extraction step S40, a region of an eye of the occupant isextracted. In the line-of-sight oscillation detection step S350, theline of sight and line-of-sight oscillation are detected from the imageof the eye region of the occupant. The result of the line-of-sightoscillation detection step S350 is sent to the object imaging step S360and oscillation comparison step S70.

In the object imaging step S360, on the basis of the line-of-sightinformation of the occupant, an object located in the line of sight isdetermined and imaged. In place of the camera, it is possible to form aLIDAR or an interferometer including a laser light source and therebydetect the oscillation from reflected light from an oscillating orvibrating object. The “LIDAR”, which is an abbreviation for LaserImaging Detection and Ranging, is a remote sensing technique thatmeasures scattered light resulting from illumination with pulsed laserlight and analyze a distance to a distant target or properties of thetarget.

In the object oscillation detection step S365, oscillation of the objectin the line of sight is detected. Oscillation of the target object in aninput image is detected. The oscillation may be detected using an imageimaged by a camera, or may be detected from information obtained byranging with laser.

In the oscillation comparison step S70, an oscillation difference iscalculated from the oscillation of the line of sight and the oscillationof the object in the line of sight. In the motion sickness estimationstep S80, a motion sickness estimation is performed from the oscillationdifference on the basis of motion sickness estimation data. In thevehicle control step S90, the vehicle is controlled on the basis of theresult of the motion sickness estimation. Also, in the motion sicknesswarning step S100, a warning of being in a motion sickness state isissued on the basis of the result of the motion sickness estimation.Then, the above flow is repeated by returning to the travelingdetermination step S10.

The above has described the operations of the motion sickness estimationdevice and motion sickness estimation method of the second embodiment.The motion sickness estimation device and motion sickness estimationmethod of the second embodiment provide the same advantages as themotion sickness estimation device and motion sickness estimation methodof the first embodiment.

Third Embodiment

FIG. 8 is a block diagram illustrating a configuration of a motionsickness prevention device including a motion sickness estimation deviceaccording to a third embodiment. A motion sickness prevention device 50includes a motion sickness estimation device 500 and a vehiclecontroller 580. The motion sickness estimation device 500 includes aline-of-sight oscillation detector 510, a velocity measurer 520, avehicle outside imaging detector 530, an object oscillation detector540, an oscillation comparator 550, a determination information storage560, a motion sickness determiner 570, and a warning presenter 590. Theline-of-sight oscillation detector 510 detects a line of sight andnystagmus (or eye oscillation) of an occupant. The velocity measurer 520measures the velocity of the vehicle. The vehicle outside imager 530obtains a range image by capturing the outside of the vehicle by usingan imaging means, such as a LIDAR, a stereo camera, or a monocularcamera. The “range image” is an image representing depth informationwith colors, monochrome shades, or the like. As described above, theLIDAR calculates a distance to a target by measuring scattered lightresulting from illumination with pulsed laser light. The stereo cameraperforms capturing with left and right two cameras, thereby generatingparallax data of the images and measuring a distance from the lenses toa target. When a monocular camera is used, a distance is calculated fromimages from multiple viewpoints captured while a camera is being moved,by using a method called Structure from Motion (SfM), for example. Inthe object oscillation detector 540, from the viewpoint of the occupantand the position of a side window, by assuming that a view area in theoutside scene that can be viewed by the occupant through the side windowis an object nearest to the vehicle, the view area is estimated. Fromthe velocity of the vehicle obtained from the velocity measurer 520 andthe distances to objects outside the vehicle obtained from the vehicleoutside imager 530, relative velocities of the objects are calculated,and when the object in the view area forms a pattern in which the objectperiodically repeats with respect to a traveling direction, it isdetermined as oscillation and the period is calculated. In general, themaximum distance that can be measured from a range image obtained usingan imaging means, such as a LIDAR or a stereo camera, is about 200meters. Since motion sickness is not likely to occur when an object orscene farther than 200 meters is viewed, it seems that detectingoscillation of an object within the range of measurement with the abovemeans is necessary and sufficient for detection of motion sickness. Aperiodic change of the object being viewed by the occupant in accordancewith movement of the vehicle is detected as oscillation by the aboveimaging means. The oscillation comparator 550 outputs a differencebetween the period of the nystagmus and the period at which the objectbeing viewed changes. The period at which the object being viewedchanges is, for example, the period at which poles or the like placed atregular intervals on the side of the road are observed from thetraveling vehicle. The determination information storage 560 storesinformation for determining whether the period of the nystagmus can leadto motion sickness. In the motion sickness determiner 570, when thedifference output from the oscillation comparator 550 is less than orequal to a threshold value and the period is greater than or equal to athreshold value in the determination information storage 560, it isdetermined that following the object outside the vehicle with the eyescan cause optokinetic nystagmus, leading to motion sickness. When it isdetermined that motion sickness can be caused by looking outside, thevehicle controller 580 blocks the outside scene from entering throughthe side window by using a light control window and darkening (orsmoking) it or by other methods, thereby preventing nystagmus. When itis determined that motion sickness can be caused by looking outside, thewarning display 590 issues a warning that suggests looking at a distantobject outside the vehicle or looking ahead of the vehicle, through aspeaker or display.

FIG. 9 is an explanatory diagram illustrating an arrangement of elementsin a vehicle including the motion sickness prevention device accordingto the third embodiment of the present invention. In the vehicleillustrated in FIG. 9 are arranged an in-vehicle camera (for detecting aline of sight and nystagmus) 610, a velocimeter 620, a vehicle outsideimaging means (LIDAR, camera) 630, an object oscillation detector 640,an oscillation comparator 650, a storage device 660, a determiner 670, aconveyance controller 680, and a speaker 690. The correspondencerelationship between the elements in FIG. 9 and the elements in FIG. 8is as follows: The in-vehicle camera 610 corresponds to theline-of-sight oscillation detector 510. The velocimeter 620 correspondsto the velocity measurer 520. The vehicle outside imaging means (LIDAR,camera) 630 corresponds to the vehicle outside imager 530. The vehicleoutside imaging means (LIDAR, camera) 630 can also be referred to as adistance measurer. The object oscillation detector 640 corresponds tothe object oscillation detector 540. The oscillation comparator 650corresponds to the oscillation comparator 550. The storage device 660corresponds to the determination information storage 560. The determiner670 corresponds to the motion sickness determiner 570. The conveyancecontroller 680 corresponds to the vehicle controller 580. The speaker690 corresponds to the warning presenter 590.

The in-vehicle camera 610 is a device that obtains line-of-sightoscillation of an occupant, which is a detection target, by imaging theoccupant. Here, among line-of-sight oscillations, nystagmus is detected.The in-vehicle camera 610 is connected to the oscillation comparator650.

The velocimeter 620 is a device that measures the velocity of thevehicle. The vehicle velocity is used when an object oscillation iscalculated. The velocimeter 620 is connected to the object oscillationdetector 640.

The vehicle outside imaging means 630 is one or more distance sensorslocated inside or outside the vehicle. Each distance sensor is a device,such as a LIDAR, a stereo camera, a monocular camera, or a radar,capable of measuring distance. The distance sensors measure distances toobjects located around and outside the vehicle, and are used fordetermining an object being viewed.

The object oscillation detector 640 determines an object being viewed bythe occupant from the vehicle velocity, the distances to the objectsoutside the vehicle, and the direction of the line of sight of theoccupant, and detects, as oscillation, the period at which the objectbeing viewed changes.

The oscillation comparator 650 calculates and outputs a differencebetween the period of the nystagmus of the occupant detected by thein-vehicle camera 610 and the period detected by the object oscillationdetector 640.

When the output of the oscillation comparator 650 is less than or equalto a threshold value, the determiner 670 compares the period of thenystagmus with a threshold value stored in the storage device 660 anddetermines whether the nystagmus has a period that can lead to motionsickness.

When the occupant is in a state where the occupant can develop motionsickness, the vehicle controller 680 blocks the view by means of smokeso that the neighborhood of the vehicle is not visible. Also, thewarning presenter 690 issues a visual or audible warning.

As described for the motion sickness estimation method of the firstembodiment, a motion sickness estimation device for executing softwareof a motion sickness estimation method is illustrated by FIG. 3 as inthe first embodiment; hereinafter, a process performed by the CPU 11will be described with reference to FIG. 10.

FIG. 10 is a flowchart illustrating a motion sickness estimation methodaccording to the third embodiment of the present invention. The motionsickness estimation method illustrated in FIG. 10 includes a travelingdetermination step S10, an estimation start signal generation step S20,an imaging step S30, a region extraction step S40, aline-of-sight/nystagmus detection step S550, a line-of-sightdetermination step S551, a distance measurement step S552, a view areaestimation step S553, a vehicle velocity detection step S554, anoscillation calculation step S555, an oscillation comparison step S70, amotion sickness estimation step S80, a vehicle control step S90, amotion sickness warning step S100, and an engine stop determination stepS110. For the reference characters of the steps of FIG. 10, the samereference characters are assigned to the steps that perform the sameoperations as those in the first embodiment and second embodiment.

The traveling determination step S10 determines whether the vehicle isin a traveling state. The determination is made by acquiringdetermination as to whether the vehicle is traveling from CANinformation or the like.

When it is determined that the vehicle is in a traveling state, in theestimation start signal generation step S20, an estimation start signalis generated by a head unit and supplied to the in-vehicle camera 610and an illumination associated therewith. Next, in the imaging step S30,the in-vehicle camera 610, which has received the supplied signal,images an occupant.

In the region extraction step S40, a device associated with thein-vehicle camera 610 extracts a region of an eye of the occupant. Inthe line-of-sight/nystagmus detection step S550, the line of sight andnystagmus are detected from the image of the detected eye region. Thenystagmus is detected from the angular velocity and period of theeyeball. On the basis of the result of the line-of-sight/nystagmusdetection step S550, in the line-of-sight determination step S551, it isdetermined whether the line of sight is directed outside the vehicle,and the degree of the nystagmus is calculated from the angular velocityof the eyeball.

In the distance measurement step S552, distances between the vehicle andobjects therearound are measured.

In the view area estimation step S553, from the viewpoint of theoccupant and the position of a side window, by assuming that a view areain the outside scene that can be viewed by the occupant through the sidewindow is the object nearest to the vehicle, the view area is estimated.

In the vehicle velocity detection step S554, the velocity of the vehicleis measured. The velocity of the vehicle is used in determining anobject being viewed by the occupant.

In the oscillation calculation step S555, from the distance from thevehicle to an object in the view area estimated in the view areaestimation step S553 and the velocity of the vehicle, a relativevelocity of the object is calculated, and when the object in the viewarea forms a pattern in which the object periodically repeats withrespect to a traveling direction, it is determined as oscillation andthe period is calculated.

In the oscillation comparison step S70, a difference between the periodof the nystagmus output from the line-of-sight/nystagmus detection stepS550 and the output of the oscillation calculation step S555 iscalculated.

In the motion sickness estimation step S80, when the output of theoscillation comparison step S70 is less than or equal to a thresholdvalue, it is determined whether the nystagmus has a period that can leadto motion sickness, by comparing the period of the nystagmus and motionsickness estimation data. When it is determined that motion sickness canbe caused, in the vehicle control step S90, a control is performed usinga light control window or a sunshade so that the neighborhood of thevehicle is not visible, or in the motion sickness warning step S100,motion sickness is improved by issuing a warning that suggests lookingin the distance or ahead of the vehicle. Then, the above flow isrepeated by returning to the traveling determination step S10.

The above has described the operations of the motion sickness estimationdevice and motion sickness estimation method of the third embodiment.The motion sickness estimation device and motion sickness estimationmethod of the third embodiment provide the same advantages as those ofthe first embodiment and second embodiment.

Although in the above description, the conveyance in which the occupantis riding has been described as a vehicle, it is not limited to avehicle, and may be any conveyance, such as a watercraft or an aircraft,that vibrates or oscillates during traveling. Although the vehicleoutside imaging means 630 has been described as measuring distances toobjects outside the vehicle, it may be an outside imaging means oroutside imager that measures distances to objects outside theconveyance.

Although embodiments of the present invention have been described above,the present invention is not limited to these embodiments.

REFERENCE SIGNS LIST

-   10 motion sickness prevention device-   100 motion sickness estimation device-   110 line-of-sight oscillation detector-   120 object oscillation detector-   130 oscillation comparator-   140 determination information storage-   150 motion sickness determiner-   160 vehicle controller-   170 warning presenter-   210 in-vehicle camera-   220 in-vehicle display with acceleration sensor-   230 oscillation comparator-   240 storage device-   250 determiner-   260 conveyance controller-   270 speaker-   30 motion sickness prevention device-   300 motion sickness estimation device-   310 line-of-sight oscillation detector-   320 object oscillation detector-   330 oscillation comparator-   340 determination information storage-   350 motion sickness determiner-   360 vehicle controller-   370 warning presenter-   410 in-vehicle camera (for detecting line-of-sight)-   420 in-vehicle camera (for imaging object in line of sight)-   430 oscillation comparator-   440 storage device-   450 determiner-   460 conveyance controller-   470 speaker-   50 motion sickness prevention device-   500 motion sickness estimation device-   510 line-of-sight oscillation detector-   520 velocity measurer-   530 distance measurer-   540 object oscillation detector-   550 oscillation comparator-   560 determination information storage-   570 motion sickness determiner-   580 vehicle controller-   590 warning presenter-   610 in-vehicle camera (for detecting nystagmus)-   620 velocimeter-   630 vehicle outside imaging means (LIDAR, camera)-   640 object oscillation detector-   650 oscillation comparator-   660 storage device-   670 determiner-   680 conveyance controller-   690 speaker

The invention claimed is:
 1. A motion sickness estimation devicecomprising: a line-of-sight oscillation detector to detect oscillationof a line of sight of an occupant by using an imaging device; an objectoscillation detector to detect oscillation of an object located in theline of sight; an oscillation comparator to calculate an oscillationdifference from the oscillation of the line of sight and the oscillationof the object; and a motion sickness determiner to determine, on a basisof the oscillation difference, whether the occupant is in a motionsickness state.
 2. The motion sickness estimation device of claim 1,wherein the object oscillation detector detects the oscillation of theobject by measuring an acceleration of the object.
 3. The motionsickness estimation device of claim 1, wherein the object oscillationdetector detects the oscillation of the object by means of an imagingdevice.
 4. The motion sickness estimation device of claim 1, wherein themotion sickness determiner uses a time integral value of the oscillationdifference in the determination.
 5. The motion sickness estimationdevice of claim 1, further comprising a warning presenter to issue awarning when the motion sickness determiner determines that the occupanthas a sign of motion sickness.
 6. A motion sickness prevention devicecomprising: the motion sickness estimation device of claim 1; and aconveyance controller, wherein the conveyance controller controls, on abasis of a determination result of the motion sickness determiner, astate of a conveyance in which the occupant is riding to reduce theoscillation difference.
 7. A motion sickness estimation devicecomprising: a line-of-sight oscillation detector to detect oscillationof a line of sight of an occupant by using an imaging device; a velocitymeasurer to measure a velocity of a conveyance in which the occupant isriding; an outside imager to capture an outside of the conveyance andmeasure a distance between the conveyance and an object located in theoutside; an object oscillation detector to detect, when the object formsa pattern in which the object periodically repeats with respect to atraveling direction in which the object is captured by the outsideimager, a period at which the object repeats with respect to thetraveling direction, as oscillation of the object, from the velocity andthe distance; an oscillation comparator to calculate an oscillationdifference from the oscillation of the line of sight and the oscillationof the object; and a motion sickness determiner to determine, on a basisof the oscillation difference, whether the occupant is in a motionsickness state.
 8. A motion sickness estimation method comprising:detecting oscillation of a line of sight of an occupant by using animaging device; detecting oscillation of an object located in the lineof sight; calculating an oscillation difference from the oscillation ofthe line of sight and the oscillation of the object; and determining, ona basis of the oscillation difference, whether the occupant is in amotion sickness state.